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ABSTRACT 

The CEAS yield iiDdel is based upon multiple r^resslon analysis at the CH) and 
state levels, Tbr the historical time series, yield is regressed on a set of 
variables derived fran mcnthly mean tonperature and mcnthly precipitation. 
Technological trend is represented by piecewise linear and/or qu^iatlc finc- 
tlons of year, indicators of yield reliability obtained from a ten-year 
bootstrap test (1970-79) demonstrated that biases are small and performance as 
Indicated by the root mean square errors are acceptable for intended 
application. However, model response for individual years, particularly unusual 
years, is not very Triable and shows some large errors. The model is 
objective, adequate, timely, simple and not costly. It considers scientific 
knowledge cn a broad scale but not in detail, and does not provide a good 
current measure of modeled yield reliability. 
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Evaluation of GEAS Mbdel 
for* Barley Yl^ds in 
North Dakota and Minnesota 

Sumnary 

The CEAS yield nodel Is based upon multiple regression analysis at the 
CRD and state levels. Bbr the historical time series, .yi^d is regressed 
on a set of variables derived from nonthly mean tonperature and nonthLy 
precipitation. Technological trend is represented by piecewise linear 
and/or quadratic functions of year. Medel performance Is evaluated on the 
basis of eight criteria - reliability, objectivity, consistency with scien- 
tific knowledge, adequacy, timeliness, cost, simplicity, and accurate 
current measures of modeled yi^d rrii^ibilityi Ten-year bootstrap t^ts 
(1970-1979) were run for each crop reporting district in the major barley 
producing regions of North Dakota and Minnesota. These indicated 
that biases are generally small and performance as Indicated by the root 
mean square errors is ^nerally acceptable for the Intended AgRISTARS large 
area ^plications. Ifowever, model response for individual years, par- 
ticularly musual years, is not very reliable and shows some large errors. 
The model is objective, adequate, timely, simple and not costly. It con- 
siders scientific knowledge on a broad scale but not in detail, and does 
not provide a g3od current neas.ure of modeled yield reliability. 

DESCRIPTION OP THE MODEL 

The model designated "CEAS Barley Model" was developed at the Center 
for Envircximental Assessment Services (CEAS) by R.P. Motha (R.P. Motha, 
"Barley Models for North Dakota and Minnesota", NQAA-CEAS, 
Columbla-Missouri, May, 1980). 



Basic inputs 1r the irodel are historical USDA yields and rtonthly rrean 
temperature and total precipitation at the Crop Reporting District (CRD) 
level. A wide variety of possible variables, such as cumulative precipita- 
tion from the previous September, monthly temperature and precipitation 
departures from normal, evapotranspiratlon (potential, actual and 
"climatically appropriate"), Z-lndex, aridity index, R-index, and noisture 
ratio are formed frcxn the basic Irputs, 

Trends, accounting for ^neral improvements in technology o/er the 
years, are defined by IVfotha as linear fron 1931 to 1962, and linear and/or 
quadratic from 196 I on. Specifically the trend ^/ariables are: 

TREND 1 = (YEAR- 1930 ) for YEAR 1 19 62 
32 for YEAR > 1962 

TREND 2 = 0.1 for YEAR 1 I 962 * 

(YEAR- 1961 ) for YEAR > 1962 

TREND 2S3 = TREND 2 ** 2 

The general form of the CEAS yield model is: 

Yi = “ + 3 ^ TREND + 3*» TREND 2 ± 

71I 

+ e » TREND 233i +K-i 

vftiere: 

Yi = estimate and yield for i-th year 
“ = intercept (constant terra) 

33’= linear trend coefficients 
3 = quadratic trend coefficient 

^ = slope coefficioit associated with the k-th weather terra 
^1K= k-th weather terra for the i-th year 


*Model was developed using these trends. Runs using TREND2 = 1.0 for 
year 1962 as one might commonly use, gave us no significant differen- 

ces in predicted yields. 
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In dsveloping the nDdels for each CRD (MN CRD’s 10 and 40, ND CRD's 10 
throu^ 90) and state (MN and ND) Motha ran st^vdse multiple r^resslons 
which examined the possible variables and selected the statistically most 
sigiif leant set of several trend and weather terns on the basis of years 
I93I-1978. A certain amount of Judgnent was used to eliminate tenns 
obviously In conflict with scientific kiowledgs (e.g., viien a coefficient 
was strongly negative vhere it should be positive) or to include important 
terms even if they v^re not statistically significant. The result was a 
set of yield nodels at CRD and state level. Appendix 2 shows the terms 
Included in each model and the range of the coeffici.ents ever the ten dif- 
ferent but overlapping sets of model base period years. 

There are some ggneral jattems tut wide diversity in detail, 
reflecting both real CRD-to-CRD variations and vagaries of the regression 
process on noisy data. 

Only aid-of-season models were tested. ALthou^ "truncated" models 
providing yield estimates at the end of each month throu^out the growing 
season were developed by Motha, it was felt that meaningful evaluation was 
difficult enou^ vhen the full-season weather was available. 

EVALUATION METHODOLOGY 
Elgdit Model Characteristics to be Discussed 
The document. Crop Yield Model Test and Evaluation Criteria , (Wilson, ^ 
al. , 1980), states: 

"The model characteristics to be emphasized in the evaluation 
process are: yield indication reliability, object ivitly, con- 
sistency with scientific knowled^, adequacy, timeliness, 
minimum costs, simplicity, and accurate current rasasures of 
modeled yield reliability." 

Each of these characteristics will be discussed with respect to the CEAS 
model . 
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Bootstrap Technique Used to Generate 
Ihcilcators of Yield Reliability 

Indicators of yield reliability require that the parameters of the 
regression nodel be computed for a set of data and that a yield prediction 
be made based on that data for a given "test" year. Ihe values required to 

.A 

generate indicators of yield relability Include the predicted yield, Y, the 
actual (reported) yield, Y, and the difference between than, d = Y-Y, for 
each test year. It is desirable that the data used to generate the parame- 
ters for the model not Include data from the test year. lb accomplish 
this, the '^Dootstrap" technique is used. For each test year, the years 
from an earlier base period are used to fit the mocfel and obtain a predic- 
tion equation. Uie values of the Independent variables for the test year 
are inserted into the .equation and predicted yield is ^nerated. Ihen, the 
last test year is added to the base period and the process is r^eated. 
Continuing in this way, ten (1970-79) predictions of yield are obtained, 
each Independent of the data used to fit the mocfel. 

A 

The Y and d values for the ten year test period are obtained from 
models derived at the crop reporting district (CRD) level and state level. 
The latter are based on a weighted average of CFD weather to the state 

A 

level. A second set of Y values are obtained at the state level using a 
weighted average of predicted jdelds from the CRD models. At the region 
level two sets of Y values are obtained, one by aggr gating CRD moctel 
yields and the other by aggregating the state model yields. In each case 
the vreighting factors are based on harvested area for the prediction year. 

For both Minnesota and North Dakota, data for 1932-1969 are used to fit 
prediction models for 1970, data from 1932-1970 are used to fit iroctels for 
1971* etc. through 1979- “Ibia testing procedure closely simulates the way 
the nodels would be applied in practice. 
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The average and .percent production and yield over the ten year test 

period are presented in Table 1 for each geogr^hlcal region. Figure 1 
’ 

presents percent production In each CRD, Itie bootstrap test results — Y, 
and d — are given in Appendix 1 for each geographical region. 

Review of Indicators of Yield Reliability 

A 

The Y, Y and d values for the ten-year test period at each ^ographic 
area nay be suiimarized into various Indicators of yi^d reliability. 

Indicators Based on d Demonstrate 
Accuracy, Precision and Bias 

Fran the d value, the mean square error ( root and relative root mean 
square error), the variance (standard deviation and relative standard 
deviation), and the bias (its square and the relative bias) are obtained. 

The root mean square error (R!VBE) and the standard deviation (SD) indi- 
cate the accuracy and precision of the model and are expressed in the ori- 
ginal units of measure ( quint als/hect are ) . It is about 68 percent pr’otable 
that the absolute value of d for a future year will be less than one RMSE 
and 95 percent probable that it will be less than twice the RMSE. rSo, 
accurate prediction capability is indicated by a small RMSE. 

A non-zero bias means the model is, on the average, overestimating the 
yield (positive bias) or mderestlmating the yi^d (negative bias). The SD 
is smaller than the RMSE when there is non-zero bias and indicates vhat the 
RMSE would be if there were no bias. If the bias is near 2 sro, the SD and 
the RMSE will be close in value. An unbiased model, i.e. bias close to 
zero, is preferred. 

Indicators Based on rd Demonstrate 
Worst and Best Performance 

The relative difference, rd (lOOd/Y), is an especially useful indicator 
in years vhere a low actual yield is not predicted accurat^y. This is 
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because years with small observed actual yld.ds and large differences haye 
the largest i?d values. 

Several indicators are derived using relative differences. In order to 
calculate the proportion of years beyond a critioal error limit, we count 
the number of years in vhlch the absolute value of the r^ative difference 
exceeds the critical limit of 10 percent. Values between 5 and 25 percent 
were investigated and a critical limit of 10 percent was found most useful 
in describing model performance. Ihe worst and next to worst perfonnance 
during the test period are (fefined as the largest and next to largest abso- 
lute value of the relative difference. Uie range of yi^d indication 
accuracy is defined by the largest and smallest absolute values of the 
relative difference. 

A 

Indicators Based on Y and Y Demonstrate 
Correspondence Between Actual and Predicted Yields 

Another set of indicators demonstrates the correspondence between 
actual and predicted yields. It would be desirable for increases in actual 
yield to be accompanied by increases in predicted yields. It would also be 
desirable for large (small) actual yields to correspond to large (small) 
predicted yields. 

Two indicators r^ate the change in direction of actual yirids to the 
corresponding change In predicted yi^ds. Che looks at change frcm the 
previous year (nine observations) and the other at change from the average 
of the previous three years (seven observations). A base period of three 
years is used since a longer base period woald further decrease the number 
of observations, viiile a storter period would not be very different fron 
the comparison to a single previous year. 
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Finally, the Pearson correlation coefficient, r, between the set of 
actual and predicted values for the test years is computed. This r^re- 
sents a measure of how veil deviations fran average in the set of pre- 
dicted yields correlate to deviations from average in the set of actual 
yields. It is desirable that r(-l 1 r 5. +1) be large and positive. A nega- 
tive r Indicates smaller predicted yields occurring with larger observed 
yields (and vice versa). 

Current Measure of Modeled Yield Reliability 
Defined by a Correlation Coefficient 

One of the nodel characteristics to be evaluated is its ability to 
provide an accurate, current measure of nodeled yield r^iability. 

Althou^ a specific statistic was not discussed in the paper. Crop Yield 
Model Test and Evaluation Criteria , (Wilson, et ^. , 1980), it was stated 
that : 

’’This ’reliability of the r^iability' characteristic 
can be evaluated by ccxnparlng model ^nerated r^iabillty 
measures with subsequenty determined deviation between 
modeled and ’true’ yield.” 

For regression models, this suggets the use of a correlation coef- 
ficient between two variables generated for each test year. One variable 
is an Indicator of the precision with vhlch a prediction for the next year 
can be made, based on the model development base period and current (test 
year) independent variable values. The other variable (obtained 
retrospectively) is an indicator of how close the predicted value for the 
next year actually is to the ’’true” value. The estimate of the standard 
error of a predicted value fran the base period model, Sy, is used for the 
first value, and the absolute value of the difference between the predicted 
and actual yield in the test year, |d | is used as the second variable. 
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Since Sy incorporates current-year yeather as compared to long-tenn. 
average, if the relation of yield to weather specified in the model is 
valid the magiitude of s^ should fluctuate in phase vdth Idl, i.e., it 
should be positively correlated. 

A non-parametric (Spearman) correlation coefficient, r, is employed 
since the assumption of bivariate normality can not be made. A positive 

A. 

value of r(-l £ r — +1) indicates agreement between Sy and Idl. An r value 
close to +1 is desirable since it indicates that a small standard error of 
prediction (and therefore a narrow confidence Interval about the true pre- 
dicted value) is associated with small discrepancies between predicted and 
actual yields. If this vere the case, one would have confidence in Sy as 
an Indicator of the accuracy of Y. 

MODEL EyALUATION 

Plots of actual and predicted ylrids for MN and ND state level nocfels 

are presented in Figure 2 and 3* Results of the tenr-year bootstrap tests 

on vhich these evaluations vere based are presented in Appendix 1. 

Indicators of Yield Reliability based on d 
Show Moderate Bias, Standard Deviations 
I^glng From 1.2 to 4.7 Q/Ha, and RMSE~ 

Rangir^ Prom 2.0 to 5»Q Q/Ha. 

A 

The indicators of yield reliability based on deviations d (= Y - Y) at 
CRD, state, and region levels are given in Table 2 and Figure 4. 

CRD level biases for ND range fran +1,6 to -1.8 Q/Ha, showing no obvious 
pattern. The biases for both MN CRD's are near -2 Q/Ha. Since the MN state 
model and aggregation to state level from CRD's both show about -2 Q/Ha 
this may indicate a general bias cn this order for the CEAS WN model. 

Root Mean Square Errors (RMSE) for ND CRD's range from 2.4 to 5.0 Q/Ha 
and for MN fran 2.6 to 4.9 Q/Ha. State level RMSE values lere somewhat 
smaller, 3-4 Q/Ha for ND and 2.9 QUa for MN. 
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Values of standard deviation ranged fran 2.2 to 4.7 Q/Ha for ND CfiD's 
and from 1.2 'to 4.4 Q/Ha for MN CRD’s. State and regional values ranged 
from 1.6 to 2.7 Q^a. 

Examination of plots of observed and predicted yields at state level in 
Figures 2 and 3 indicates that in both ND and MN the CEAS node! predictions 
seem to be biased by a consistent -2 Q/Ha in the years 1975-1979* liiis may 
indicate a wealmess In the CEAS model and is discussed in the conclusions 
section. 

Indicators of Yield Reliability Based on rd show 
that a Large Number of Cases Have 50 Percent or 
More of Test Yeairs vdth rd Greater Than 10 Percoit 

The CRD, state, and region values for the Indicators of yield reliabi- 
lity based on absolute values of relative differences, 1 rd I, are given in 
Table 3 and Figures 6, and 7* 

Seven of the nine ND CRD’s and both MN CRD's show 50^ or more of the 
test years with rd greater than 10^. State and regional results show two 
of six cases with 50^ or more of the test years with I rd I greater than 10^. 
These results would seem to Indicate either a large natural variability in 
barley yields or a low level of model skills. Both indications are sup>- 
ported by the plots in Figures 2 and 3* If the model capabilities could be 
slgiiflcantly Improved In the years 1975-1979 the indicators of yield 
reliability would also be much Improved. 

For ND 1974 was the year with the largest relative difference in eight 
of nine CRD's. All nine represented an inability of the model to respond 
to a very low actual yield. For MN 1976 and 1977 were the worst years, 
representing unde res tlmat Ions by the model of high actual yields. 
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Indicators of Yield Reliability Based on Y and Y 
Show Moderately Good Correspondence Betweoi the 
Direction of Change In Predicted Yield Compared 
to Actual Yield . 

The predicted and actual yields at state level are plotted in Figures 2 
and 3. Ihe predicted yields, actual yields, and differences for CRD level 
are listed in Appendix 1. Ihe CRD, state, and region level values for 
Indicators of yield reliability based on actual and predicted yields are 
given In Table 4 and Figures 8, 9, and 10. 

Out of the nine ND and two WN CRD's, six show a change of direction of 
predicted yields from the previous year corresponding to the actual change 
of direction more than 50 percent of the time. When the base period is the 
average of the previous three years the score is nine of eleven CRD's 
correct nore than 50 percent of the time. Bbr state and regional models 
the response direction fhora the previous year is correct more than 50 per- 
cent of tile time In five of six cases, and the response direction frcm the 
three year average Is correct in all six cases. These results Indicate that 
the CEAS model does reasonably well In responding to changes In actual 
yield, particularly changes from a three-year base period. 

Results for the correlation coefficient, r, between predicted and 
actual yields, representing correlation between fluctuations of predicted 
and actual yields from averages over the test period are not very 
satisfactory. Of the eleven CRD's only four show r greater than . 55 , the 
one-tailed value required for statistical significance. Values of r as low 
as 0.05 and 0.17 are found, and six of eleven are less than .30. The score 
for state and regional nodels Is four of six g[*eater than . 55 . Clearly the 
directional response capabilities of the model leave much to be desired. 
This is especially true for those specific cases of large actual fluc- 
tuations (see Figures 1 and 2). 


• 10 - 


Base Period Ihdlo^es More Precision Than 
Independent Iteats Can Conf £m 

Certain statistics g3nerated from the regression analysis of the base 
period data are often used to provide some indication of expected yi^d 
reliability. However* these statistics only reflect how \-^ll the model 
describes the data used bo generate the nodel, i.e,, fit of the model, 
rather than how v^l the model can predict given new data, therefore, it 
is Important to compare these Indicators of fit of the model to the inde- 
pendent Indicators of yield rdLiability discussed in the preceding sections. 
In this way, one can see how these base period indicators of fit of the model 
do or do not correspond to Independent test Indicators of yield r^i ability. 

One Indicator of yield reliability, the msan squ'ire error (MSE), is the 

A 

sum of squared d values (d « Y - Y) for the independent test years divided 
by the number of test years (Table 5). Uie direct analogue for the model 
developmsnt base period is the residual mean square. The residual mean 
square is obtained by first generating the usual least squared prediction 
equation using the base period years. Then instead of predicting the yield 
of the fbllowing test year, yields are predicted for each of the base 
period years. The residual mean square is the sum of squared d values 
for these base period years divided by the appropriate degrees of freedom 
(number of base period years minus number of parameters estimated in 
fitting the model). Whereas one value of MSE is generated for each 
geographic area over the oitire test period, a value of the residual mean 
square is ^nerated for each base period corresponding bo an individual 
test year. 

Another indicator of yi^d rd-iabHity is the correlation coefficient, 
r, between the observed and predicted yields for the Independent test jrears 
(Table 6). It is desirable for r to be close to +1, even thou^ it can be 
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negative. Ihe analogue fop the nodel development base period Is the square 
root of r 2 expressed as a proportion, R(0~ Ri. 1). It can be Interpreted 
as the correlation between observed and predicted yield. Values of R for 
each geographic area are given in Table 6, along vdth the Pearson corrria- 
tlon coefficient values from Table 4. 

Average correlation coefficients over the base period (model develop- 
ment years) range from .88 to .97, Indicating the nodel is doing a very 
good job of fitting the development data. The correlation coefficients 
over the Independent test years range from lows of .05 to .17 to highs of 
.79 and .95. ihe average of r over the independent test years is around 
.43, less than half the r for the node! development years. CJlearly the 
CEAS model does not respond nearly as well in a predictive mode as it does 
in a fitting nocfe. Essentially, the values of R for model oevelopnent 
years provide no Indication of the predictive abilities of the model. 

Model is Reasonably Objective 

The nature of the CEAS model requires that it be redeveloped (i.e., 
coefficient values re-derived) for each test year, based on available years 
prior to it. Once the proper terms have been sheeted and fixed, develop- 
ment and £?)pllcatlon of the model is quite objective. A great deal of 
subjectivity, however, is required for initially selecting the terms, in 
specifying trend, particularly break points, and in choice of development 
years. 

Model Considers Kiown Scientific 
Relationships on a Broad Scale 

Selection of model terms is by st^wise regression. Hiis guarantees 

only the set of terms "best" by some statistical criterion. Ehyslcal 

slgiiflcance is not ensured. It seems mlikely that the wide variety of 

"significant" terms represented In Appendix 2 for different CRD models is , 
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entirely meaningful in a physical sense, ahe selection criteria used by 
the no del author are not well described. 

Large-area crop yields are known to be related to weather over the 
growing season, to pceseason stored soil iroisture, and to a variety of 
other weather and agronomic factors . Ihe (fetaHs of the nathematical rela- 
tionships that describe these physical relationships are far from 
established. Bren the proper set of variables is open to qiestion because 
there are only a few readily available observables and the variables formed 
from these tend to be highly Interrelated. Large-area relationships are 
further confused by geographical variations in the observables that may or 
may not be lmpor»tant for any given situation. 

In li^t of these p?oblems, the authors of the CEAS models choose to rely 
on a practical spproach of statistical r^ression of observed yi^ds to 
monthly v^ather data, trend represented as a function of historical years, 
and a policy of refitting for each predictive year based on all available 
prior years. 

Thus, the CEAS model is susceptable to criticism in regard to agreement 
with scientific knowledge in many respects. A few of the mere Important 
are noted below. The CEAS model handles technology and cropping prac- 
tice trends by representing them as piecewise linear and/or quadratic func- 
tions of time. This fosses over the known qualitative relationships to 
variety Improvement, fertilizer use, etc., but represents a practical way 
of treating the situation viiere it is unclear vhich effects are most impor- 
tant and vhere information is limited. Rationale for choosing breakpoints 
between trend segments or for specifying linear or quadratic s^ments seams 
to be primarily on a practical rather than a scientific basis. The CEAS 
model takes no explicit account of pests, disease, or other ^isodic 
events. 
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Model is Adequate Ctily for the Region 
Jn Vftiich It Was Developed 


By its nature, a given CEAS model can be applied vdth any degree of 

reliability only in the region for vhlch it was developed. Ihe CEAS models 

are not extendable even to apparently similar regions. Cn the other hand, 

the CEAS approach can be readily ^plied to any region vhere a reasonably 

lengthy record (say 15-20 years) of yield and weather observations exist. 

Model Is limely Enourii Pbr 
Intended Applications 

A yield model for a new year can be built as soon as reliable yield and 
weather variable figures fran the past year are available, in the U.S. 
generally a few months, after harvest. In foreign countries a longer period 
of time. Yield predictions during an ^pLicatlon .vear can be made shortly 
after the end of each nonth for v4iich vreather data is available. 

Model is Not Costly 

Data to develop and run the CEAS barley model are readily available at 
low cost. Ihe multiple regressions needed to compute the agroncrnic and 
meteorological variables and develop models can be run on any modest size 
computer. Routines are available in most computer libraries. 

Model is Simple 

The development and application of the CEAS model are straightforward. 

The only points vhere judgment is required are in selection of significant 
terms, specification of trend, and estimation of soil moisture bud^t capacity. 

Model Has Poor Current Measure of 
Modeled Yield Reliability 

The CRD, state, and region values of the corrd.atlon confident bet- 
ween the estimate of the standard error of the predicted yield values and 
the absolute differences between predicted and actual yidd are presented 
in Table 7 and Figure 11. The results are very poor. In 11 of 13 cases 





the correlation is negative, and the largest positive value is 0.06. 
Clearly the model does not provide a good estimate as to how close the pre- 
dicted yields vdll be to actual yields in any prediction year. 

CONCLUSIONS 

The CEAS yield model for barley represents a straightforward multiple 
regression fit of piecewise linear and/or quadratic trend and the most 
significant weather-r^ated tenns available for prior years. Pits are made 
at CRD and state levels. The data bases consist of USDA observed yields 
and monthly values of mean temperature and total precipitation. Indicators 
of yield reliability obtained from a ten year bootstrap test (1970-1979) 
are used to evaluate the model. 

Over the set of test years the reliability of the model on average is 
Indicated to be acceptable for many applications. Root mean square erron 
are about 3 Q/Ha. The CEAS model does not consistently predict high or low 
actual yields very accurately, and for any .given year the actual error may 
be appreciably larger than the RMSE value. The model does not give a good 
current measure of yield reliability. However, it is objective, adequate 
for Intended purposes, timely, simple, not costly, and makes a practical 
attsnpt at incorporating some general scientific kiowled^. 

Many general areas of needed Improvement could be cited. The most 
obvious specific improvement would be to correct the consistait Was of the 
CEAS model in ND and MN in 1975-1979* ®iis vjould considerably Improve RMSE 
and other indicators of reliability. A fit mde with the TREND2S3 tenn 
ronoved, leaving linear trend segments 1931-1961 and 1962-1979j gave pre- 
dicted yields coinciding almost exactly with actual yields in 1975-1979 but 
much poorer correspondance In 1970-1974. Across the ten year test period 
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the RMSE for this fit ms slightly worse than that for the original CEAS 
model. Clearly, the fix is not such a simple adjustment. 

One final note on the CEAS models should be added. During the several 
years of testing conducted for the Large Area Crop Inventory Experiment 
(LACIE) and subsequent years prior to AgRISTARS no yield model was found to 
outperform the CEAS models. 
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Figure 1: ^ -crley by CRu '0*1979 average, as a percer.t of tnt 

regional tocal. Darker shades Indicate CRD's with higher production. 
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Figure 2 

Actual and Predicted 
Yields for North Dakota 
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TABLE 2 

^INDICATORS OF YIELD RELIABILITY 
BASED ON D = PREDICTED - ACTUAL YIELD 

CEAS MODEL - BARLEY 
NORTH DAKOTA AND MINNESOTA 

MSE» VAR, B-SQR (OUINTALS/HECT ARE SQUARED) 
^RMSE* SD,^BIAS (QUINTALS/HECTARE) 
RRMSE» RSD, RB (PERCENT OF AVERAGE YIELD) 


STATE 

CRD 

MSE 

RMSE 

RRMSE 

VAR 

SD 

RSD 

B-SQR 

bias 

RB 

N. DAKOTA 10 

8,95 

2.99 

15.4 

6,36 

2.52 

12.0 

2.59 

1.61 

8,3 


20 

11,53 

3o39 

17.9 

9.84 

3.14 

15.5 

1.69 

1.30 

6,9 


30 

5,78 

2.40 

11.0 

4.88 

2.21 

9.7 

0,90 

0.95 

4,3 


40 

25,05 

5,01 

25.8 

21.78 

4.67 

26.5 

3.28 

-1 .81 

-9,3 


50 

i b , 8 7 

4.11 

21.8 

14.62 

3.B2 

22.1 

2.25 

-1.50 

—8 . 0 


60 

10.65 

3.26 

14,0 

10.08 

3.18 

13.2 

0,56 

0.75 

3.2 


70 

15,17 

3.90 

20.6 

14.78 

3.84 

21.0 

0.40 

-0.63 

-3,3 


80 

13,23 

3,64 

22 . / 

13,06 

3.61 

23,2 

0.17 

-0,41 

-2,6 


90 

12.94 

3.60 

18,, 0 

11.38 

3.37 

18.0 

1.56 

-1.25 

-6,2 

STATE 

MODEL 

11.22 

3.35 

15.9 

7.38 

2.72 

14.3 

3.84 

-1,96 

-9,3 

CRDS 

AGGR. 

5.93 

2.44 

11.6 

5.77 

2.40 

11.2 

0.16 

0.40 

1.9 

MINNESOTAIO 

6.49 

2,55 

10.2 

1.47 

1.21 

5.4 

5.02 

-2.24 

-9,0 


40 

23,94 

4,89 

21.9 

19,58 

4.42 

21.9 

4.37 

-2.09 

-9.4 

STATE 

MODEL 


2.89 

12.1 

4.93 

2,22 

10.0 

3,42 

-1.85 

-7,7 

CRDS 

AGGR • 

7.00 

2.65 

11.0 

2.63 

1.62 

7.4 

4.37 

-2.09 

-8,7 

REGION 

CRDS 

AGGR, 

4.38 

2.09 

9.6 

4.30 

2.07 

9.6 

0.08 

-0,28 

-1,3 

STATES 

AGGR, 

9,75 

3.12 

14.3 1 

5.95 

2.44 

12.3 

3.80 

-1.95 

-8.9 
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TABLE 3 

INDICATORS OF YIELD RELIABILITY 

100 » ((PREDICTED-ACTUAL YIELD) /ACTUAL YIELD) 

CEAS MODEL - BARLEY 
NORTH DAKATO AND MINNESOTA 


1 

STATE CRO 1 

PERCENT 
OF years 
IRDI >10% 


LARGEST IRDI \ 

RO (YEAR) 1 
^ 1 

NEXT 

LARGEST 


RANGE 

IRDI 

1 

N. DAKOTA 10 | 

40 


44,9 

— — 1 
(1974) 1 

30.7 

0,5 

44.4 

20 1 

70 


68.9 

(1974) ! 

21.5 

0.0 

68.9 

30 1 

30 


38.5 

(1974) 

14.4 

-1.2 

37.3 

40 1 

80 


71.3 

(1974) 

-38.1 

0.4 

71.4 

50 1 

70 


41.5 

(1974) 

36.6 

0.6 

40.9 

60 1 

60 


29.7 

(1970) 

25,0 

-28.5 

-1.7 

28.1 

IS I 

60 


46.4 

(1974) 

2.4 

44.0 

80 1 

90 


58,4 

(1974) 

-26.6 

-4.7 

53.7 

90 1 
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-27.9 

1.1 
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70 


26.5 

(1974) 
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-1.4 

25.1 

CRDS AGGR.I 
1 

30 


39.1 

(1974) 

13.7 

-1.0 
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50 
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13.7 
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Figure 6: Largest absolute value of the relative difference for barley during the 

test years 1970-1979. Darker shades indicate CRD’s with higher production. 
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Figure 7: Next largest absolute value of the relative difference for barley during 

the test years 1970-1979. Darker shades indicate CRD's with higher 
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TABLE 4 

INDICATORS OF YIELD RELIABILITY 
BAS^D on ACTUAL AND PREDICTED YIELDS 


CEAS MODEL - BARLEY 
NORTH DAKOTA AND MINNESOTA 


STATE CRD 

PERCENT OF YEARS 1 

DIRECTION OF CHANGE IS CORRECT 1 PEARSON 

FROM PREVIOUS YEARI FROM BASE RERIOD 1 CORR. COEF. 

1 1 

N. DAKOTA 10 


67 


43 

1 — — — 

1 0.57 

20 


33 


36 

1 0.05 

30 


78 


71 

1 0.79 

40 


56 


57 

1 0.17 

50 


56 


57 

1 0.29 

60 


56 


71 

1 0.57 

70 


44 


43 

1 0.23 

80 


44 


57 

1 0.27 

90 


44 


71 

! 0.42 

1 

STATE MODEL 


33 


57 

1 0.35 

CRDS AGGR. 


67 


71 

1 0.57 

1 

MINNESOTA 10 

t 

78 


100 

I 0.95 

40 


33 


71 

1 0.18 
1 

STATE MODEL 


56 


96 

1 0.69 

CRDS AGGR. 


67 


96 

I 0.86 

1 

REGION 





1 

1 

CRDS AGGR. 


78 


71 

1 0.73 

STATES AGGR. 


67 


71 

1 0.48 
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Figure 9: Percent of test years (1970-1979) the directions of change of predicted yield 

from the previous three year average agrees with the direction of change in 
actual barley yield. Darker shades indicate CRD's with higher production. 
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TABLE 5 

, RESIOUAL MEAN SQUARE AS AN 
^INDICATOR OF THE FIT OF THE MODEL 
BASED ON THE MODEL DEVELOPMENT BASE OERIOD 

CEAS MODEL - BARLEY 
NORTH DAKOTA AND MINNESOTA 


1 

STATE CRD 

^ RASE PE 
RESIDUAL ME 
LOW HIGH 

RIOD 

AN SQUARE 
AVERAGE 

IVJOEPENDENT 

TEST 

MSE 

N. DAKOTA 10 

1 

1 5,33 

6.29 

5.78 

8.95 

20 

5.12 

6.37 

5.76 

11.53 

30 

2.63 

3.21 

2.90 

5.73 

40 

3.63 

5.63 

4.81 

25.05 

50 

4.73 

5.99 

5.20 

16.87 

60 

2.02 

3.07 

2.71 

10.65 

70 

4.58 

5.69 

5.20 

15.17 

80 

3.54 

4.24 

3.94 

13,23 

90 

2»4.0 

2.76 

2 . 5S 

i 12.94 

State model 

1.99 

2.77 

2.25 

11.22 

MINNESOTAIO 

1.58 

1.89 

1.68 

6.49 

40 

5.21 

7.03 

5.74 

23.94 

STATE model 

3.26 

3.79 

3.56 

8.35 

I 
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CORRELATrO^J BETWEEN OBSERVEb^AND PREDICTED YIELDS AS AN 
^ ^INDICATOR OF THE FIT OF THE MODEL 
BASED ON THE MODEL DEVELOPMENT BASE PERIOD 

CEAS MODEL - BARLEY 
NORTH DAKOTA AND MINNESOTA 


TEST 

STATE CRD 

BASE, PERIOD 
CORRELATION COEF, 
LOW HIGH AVERAGE 

INDEPENDENT 
CORR. COEF, 

N. DAKOTA 10 

1 

1 0.91 

0.93 

0.92 

0,57 

20 

1 0.87 

0.90 

0.88 

0,05 

30 


0.95 

0.94 

0,79 

40 

1 0.92 

0.94 

0,93 

0,17 

50 

1 0.91 

0.93 

0,92 

0,29 

60 

1 0.94 

0.96 

0.95 

0.57 

70 

1 0.92 

0.93 

0.93 

0.23 

80 

1 0.93 

0.94 

0.93 

0.27 

90 

; 0.96 

0.97 

0,97 

0.42 

STATE MODEL 

0.95 

0.96 

0.96 

0,35 

MINNESOTAIO 

0.97 

0.97 

0,97 

0.95 

40 

0.91 

0.93 

0,92 

1 0.18 

STATE MODEL 

0.91 

0.94 

0.93 

0.69 

1 
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table 7 

CURRENT INDICATION OF 
MODELED YIELD RELIABILITY 

AGREEMENT BETWEEN BASE PERIOD PREDICTED 
AND TEST YEAR ACTUAL ACCURACY 

CEAS MODEL - BARLEY 
NORTH DAKOTA AND MINNESOTA 


STATE CRD 

spearman 

correlation COEF. 

N. DAKOTA 10 

-0.37 

20 

-0.15 

30 

-0.12 

40 

-0.81 

50 

-0.46 

60 

-0.54 

70 

-0,09 

80 

0.06 

90 

0.01 

STATE MODEL 

-0.35 

MINNESOTAIO 

-0.30 

40 

-0.52 

STATE MODEL 

-0.55 
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ORIGINAL PAGE SS 





OF POOR QUALITY 





APPENDIX 1 





BOOTSTRAP 

TEST RESULTS 




FOR 

BARLEY YIELDS 

IN 




NORTH 

DAKOTA 

AND MINNESOTA 



USING A CEAS TREND 

AND monthly WEATHER 

DATA 

MODEL 



YIELD 

(Q/H) 



S.E. 

STATE CRD 

YEAR ACTUAL 

PREO. 

0 

RD 

PRED. 

N. DAKOTA 10 

1970 

20.1 

23.5 

3,4 

16.9 

2.86 


1971 

20,9 

22,1 

1.2 

5.7 

2,81 


1972 

21.0 

23.1 

2.1 

10.0 

2.73 


1973 

22.5 

21.2 

-1.3 

-5.8 

2.73 


1974 

13,8 

20.0 

6.2 

44.9 

2.69 


1975 

16.6 

21.7 

5.1 

30.7 

2.84 


1976 

19.0 

19.1 

0.1 

0.5 

2.90 


1977 

18.6 

17.7 

-0.9 

-4.3 

2.79 


1978 

25.0 

23.5 

-1.5 

-6.0 

2.73 


1979 

16,9 

18.6 

1.7 

10.1 

2.71 

20 

1970 

18.6 

22.6 

4.0 

21,5 

2.78 


1971 

21.4 

21.5 

0.1 

0.5 

2,80 


1972 

20,6 

20.6 

OaO 

0,0 

2,70 


1973 

20.4 

22.5 

2.1 

10.3 

2.63 


1974 

12.2 

20,6 

8.4 

58.9 

2.61 


1975 

17.7 

19.6 

1.9 

10.7 

2.88 


1976 

19.8 

17.7 

-2.1 

-10.6 

2.87 


1977 

16,4 

18.4 

2.0 

12.2 

2.90 


1978 

22.5 

19.0 

-3.5 

-15.6 

2.81 


1979 

19.7 

19.8 

0.1 

0,5 

2.75 

30 

1970 

19.5 

21.6 

2.1 

10,8 

2.04 


1971 

24.5 

25.1 

0.6 

2.4 

2.07 


1972 

21.9 

23.1 

1.2 

5.5 

2,04 


1973 

20.1 

23.0 

2.9 

14.4 

1.89 


1974 

14.8 

20.5 

5.7 

38.5 

2,17 


1975 

22.7 

21.3 

-1.4 

-6.2 

2.14 


1976 

22.3 

21.6 

-0.7 

-3.1 

2.06 


1977 

21.8 

23.5 

1.7 

7.8 

1.97 


1978 

24,4 

24.1 

-0.3 

-1.2 

1.95 


1979 

27.2 

24,9 

-2.3 

— 8,5 

1.92 

40 

1970 

17,1 

19.9 

2.8 

16.4 

3.19 


1971 

21,5 

13.3 

-8.2 

-38.1 

3,00 


1972 

23,9 

24,0 

0.1 

0,4 

3,23 


1973 

20.8 

14.4 

-6.4 

-30,8 

3.13 


1974 

11.7 

20.1 

8.4 

71.8 

3.06 


1975 

17.4 

18.1 

0.7 

4.0 

3.27 


1976 

19.9 

14.7 

-5.2 

-26.1 

3.08 


1977 

16.7 

14.1 

-2.6 

-15.6 

3.35 


1978 

25.5 

21,6 

-3 . 9 

-15,3 

3.13 


1979 

19.6 

15.8 

-3.8 

-19.4- 

3.32 
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APPENDIX 1 

BOOTSTRAP TEST RESULTS 
FOR barley yields IN 

\ CEAS trend and monthly ■■■ 


:nnesota 

WEATHER 


DATA MODEL 


CRD 


50 


60 


70 


80 


YEAR 

YIELD 

ACTUAL 

(Q/H) 

PRED. 

D 

RD 

S.E. 

PRED, 

1970 

17.7 

17.8 

0.1 

0.6 

3,65 

1971 

1972 

24,5 

20«4 


-3.2 
-1 .4 

-ii:i 

3.38 

3,30 

1973 

14.5 

19.8 

5.3 

36.6 

3,16 

1974 

12.3 

17.4 

5.1 

41,5 

3,46 

1975 

19.9 

14.6 

-5.3 

-26.6 

3.2A 

1976 

18.3 

15.4 

-2.9 

-15.8 

3,07 

1977 

16.7 

11.2 

-5.5 

-32.9 

3.13 

1978 

22,9 

17.1 

-5.8 

-25,3 

3,09 

1979 

20,9 

19.5 

-1.4 

-6.7 

3.15 

1970 

17.5 

22.7 

5.2 

29.7 

1.81 

1971 

26.5 

25.2 

-1.3 

-4,9 

1.99 

1972 

22.6 

25.0 

, 2.4 

10.6 

1.85 

1973 

C 1 . J 

25.0 

3.7 

17, A 

1,81 

1974 

18.4 

23.0 

4,6 

25.0 

1 .9A 

1975 

21.5 

20.9 

- 0.6 

- 2.8 

2,11 

1976 

22.8 

24.7 

1.9 

8.3 

2,02 

1977 

24.1 

23.7 

-0.4 

-1.7 

2.02 

1978 

28.6 

25.3 

-3.3 

— 1 1 . D 

1.90 

1979 

29.3 

24.6 

-4,7 

-16.0 

2.07 

1970 

16.4 

16.8 

0.4 


3,79 

1971 

21.6 

15.5 

- 6 . 1 

-28,2 

3,27 

1972 

21 .4 

25.1 

3.7 

17.3 

3.47 

1973 

22.1 

18.0 

-4.1 

-18.6 

3.19 

974 

15.3 

22.4 

7.1 

A6.A. 

3.12 

1975 

16.9 

16.2 

-0.7 

-4.1 

3,38 

1976 

19,6 

17,0 

- 2.6 

-13.3 

3,22 

1977 

17.2 

12.3 

-4.9 

-28.5 

4,13 

1978 

20.8 

22.2 

1.4 

6.7 

3,25 

1979 

17.7 

17.2 

-0.5 

- 2.8 

3.05 

1970 

13,0 

16.4 

3.4 

26.2 

3.43 

1971 

21.4 

15.7 

-5.7 

-26,6 

2,94 

1972 

18.9 

22.1 

3,2 

16.9 

3.00 

1973 

16.3 

14.4 

-1.9 

-11.7 

2.87 

1974 

10.1 

16.0 

5.9 

58. A. 

2.72 

1975 

17.8 

14.6 

-3.2 

-18,0 

3,08 

1976 

14.2 

16.0 

1.8 

12,7 

2.92 

1977 

12.7 

12.1 

- 0.6 

-4.7 

2.83 

1978 

19.0 

16.3 

-2.7 

-14,2 • 

2.65 

1979 

16.5 

12.2 

-4.3 

-26.1 

2.65 
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USING A CEAS 
STATE CRD 


N. DAKOTA 90 


STATE MODEL 


CROS AGGR. 
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OF POOR QUALITY 




APPENDIX 3, 

BOOTSTRAP TEST RESULTS 
FOR BARLEY YIELDS IN 
NORTH^OAKOTA AND MINNESOTA 
TREND AMD MONTHLY WEATHER DATA MODEL 


YIELD (0/H) 


S.E. 


YEAR 

ACTUAL 

PRED. 

D 

RD 

PRED. 

1970 

18,5 

17.5 

-1.0 

-5.4 

2.82 

1971 

24.8 

20.9 

-3.9 

-15.7 

2.47 

1972 

21.3 

23.1 

1.8 

8.5 

2.54 

1973 

18.7 

18.9 

0.2 

1.1 

2.40 

1974 

17.1 

23.4 

6.3 

36.8 

2.82 

1975 

17.8 

15.5 

-2.3 

-12.9 

2.45 

1976 

13.8 

13.4 

-0.4 

-2.9 

2.57 

1977 

23.1 

19.1 

-4.0 

-17.3 

2.60 

1978 

22.3 

19.5 

-2.8 

-12.6 

2.35 

1979 

22.9 

16.5 

-6.4 

-27,9 

2.37 

1970 

18.3 

19.7 

1.4 

7.7 

2.43 

1971 

24.2 

19.9 

-4.3 

-17.8 

2.19 

1972 

21.5 

21.2 

-0.3 


2.30 

19/3 

19.9 

18.5 

-1,4 

-7,0 

2.12 

1974 

15.1 

19.1 

4.0 

26.5 

2.36 

1975 

20.4 

16.5 

-3.9 

-19.1 

2.19 

1976 

20.4 

17.6 

-2.8 

-13.7 

2.16 

1977 

21.0 

17.2 

-3.8 

-18.1 

2.04 

1978 

24.7 

20.2 

-4.5 

-18.2 

2.07 

1979 

24.7 

20.7 

-4.0 

-16.2 

2.10 

1970 

18.3 

20,8 

2.5 

13.7 


1971 

24.2 

23.0 

-1-.2 

-5.0 


1972 

21.5 

22.9 

1.4 

6.5 


1973 

19.9 

21.5 

1.6 

8.0 


1974 

15.1 

21.0 

5.9 

39.1 


1975 

20.4 

19.4 

-1.0 

-4.9 


1976 

20.4 

20.1 

-0.3 

-1.5 


1977 

21.0 

20.8 

-0.2 

"1.0 


1978 

24.7 

22.6 

-2.1 

-8.5 


1979 

24.7 

22.1 

-2.6 

-10.5 
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appendix 1 

BOOTSTRAP TEST RESULTS 
FOR BARLEY YIELDS IN 
DAKOTA AND MINNESOTA 

USING A CEAS TREND AND MONTHLY WEATHER DATA MODEL 


STATE CRD YEAR ACTUAL ^PRED. 


D 

RO 

-0.1 

-0.5 

-3.1 

-11.5 

-0.6 

-2.4 

-1.5 

-6.2 

-2.1 

-10.0 

-1.9 

-8.8 

-3,8 

-14,2 

-3,3 

-12,0 

-2.2 

-7.7 

-3,8 

-12.9 

-2.2 

-9.6 

0.6 

2.4- 

2.0 

10,4. 

-6.3 

= 24.0 

0,1 

0.5 

-0.6 

-3,2 

4.8 

36,1 

•10.7 

-38.6 

-1.5 

-6.8 

-7,1 

-26.9 

0.9 

4.5 

-2.6 

-10.0 

0.7 

3.0 

-1.9 

-7.7 

0.5 

2.4 

-0.4 

-2.0 

-1.9 

-8.6 

-5.9 

-21.5 

-3.0 

-11.3 

-4,9 

-17.2 

-0.8 

-4.0 

-1.8 

-6.8 

0.1 

0.4 

-2.9 

-11.7 

-1,3 

-6.2 

-1.4. 

-6 . 8 

-0.9 

-4,1 

-5.3 

-19.3 

-2,0 

-7.5 

-4,6 

-16.0 


S.E. 


MINNESOTA 10 


AO 


STATE MODEL 


CRDS AG6R. 


1970 

1971 

1972 

1973 
197A 

1975 

1976 

1977 

1978 

1979 


1970 

1971 

1972 

1973 

1974 

1975 

1976 

1977 

1978 

1979 


1970 

1971 

1972 

1973 

1974 

1975 

1976 

1977 

1978 

1979 


1970 

1971 

1972 

1973 

1974 

1975 

1976 

1977 

1978 

1979 


18.4 
26.9 

24.8 
24.2 

20.9 

21.5 
26.7 

27.4 

28.4 

29.4 


22.9 

25.2 

19.3 

26.3 
21.2 
18.6 

13.3 
27.7 
22.2 

26.4 


19.9 

26.1 

23.1 
24.7 
21.0 

20.4 

22.1 

27.4 
26.6 

28.5 


19.9 

26.3 

23.3 

24.8 

21.0 

20.5 

22.2 

27.5 

26.8 
28.7 


18.3 

23.8 

24,2 

22.7 

18.8 

19.6 
22.9 

24.1 

26.2 

25.6 


20.7 

25.8 
21.3 
20.0 

21.3 
18.0 
18.1 
17.0 
20.7 

19.3 


20.8 

23.5 

23.8 

22.8 

21.5 
20.0 
20.2 

21.5 

23.6 
23.6 


19.1 

24.5 

23.4 

21.9 

19.7 

19.1 
21.3 

22.2 

24.8 
24.1 


2,42 

1,99 

1.88 

1.82 

1.89 

1.75 

1.84 

1.76 

1.85 
1.84 


4,14 

3.51 

•a *30 

^ t C 

3.49 

3.30 

3.23 

3.13 

3.51 

3.43 


3.53 
3.03 
2.83 
2.72 
2.79 
2.57 

2.53 
2.41 
2.52 

2.54 
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^OOR QUA14TY 


APPENDIX 1 

BOOTSTRAP TEST RESULTS 
FOR BARLEY YIELDS IN 
. ^ NORTH DAKOTA AND MINNESOTA 

USING A CEAS TREND AND MONTHLY WEATHER DATA MODEL 


YIELD (Q/H) 

STATE CRD YEAR ACTUAL PRED, 


RO 


S.E. 

PRED. 


REGION 

CRDS AGGR. 


1970 

1971 

1972 

1973 

1974 

1975 

1976 

1977 

1978 

1979 


18.6 

24.8 

21.9 
21.1 
16.6 
20.5 

20.9 

22.9 
25.4 
26.0 


20.4 

23.4 
23.0 
21.6 

20.7 

19.3 

20.4 
21.2 
23.2 

22.7 


1.8 

■1.4 

0.5 

4.1 

• 1.2 

•0.5 

•1.7 

■ 2.2 

•3.3 


9.7 

-5.6 

5.0 

2.4 

24.7 

-5.9 

-2.4. 

-7.4 

-8.7 

-12.7 


STATES AGGR. 


1970 

1971 

1972 

1973 

1974 

1975 

1976 

1977 

1978 

1979 


18.7 

24.7 

21.9 

21.1 

16.7 
20.4 

20.9 

22.9 
25.3 

25.9 


20.0 

20.9 

21.8 

19.5 
19.7 

17.5 
18.3 

18.5 

21.2 

21.6 


1.3 

•3.8 
• 0.1 
• 1.6 

3.0 

•2.9 

• 2.6 

•4,4 

•4.1 

■4,3 


-15,4 

-0,5 

-7,6 

18.0 

-14,2 

-12.4 

-19,2 

-16.2 

-16.6 
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APPENDIX 2: Terras and Range of Values of Coefficients 
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